Run the full stage 2 `run-make` test suite in `x86_64-gnu-debug` #131917

jieyouxu · 2024-10-19T07:26:21Z

Run the full run-make test suite in the x86_64-gnu-debug CI job. This is currently the only CI job where //@ needs-force-clang-based-test will be satisfied, so some run-make tests will literally never be run otherwise. Before this PR, the CI job only ran run-make tests which contains the substring clang in its test name, which is both (1) a footgun because it's very easy to forget and (2) it masks tests that would otherwise fail (even failing to compile) because the test is skipped if doesn't have a clang in its test name.

With the environment of x86_64-gnu-debug, two run-make tests failed before this PR:

tests/run-make/issue-84395-lto-embed-bitcode/rmake.rs: this was broken for a long time because objcopy in llvm bin tools was renamed to llvm-objcopy. This test was converted into a rmake.rs test, rather straight forward.
tests/run-make/cross-lang-lto-riscv-abi/rmake.rs: this was broken for a long time and never worked. The old version inspected human-readable output of llvm-readobj --file-header looking for substring EF_RISCV_FLOAT_ABI_DOUBLE, but the human-readable output will only contain something like Flags: 0x5, RVC, double-float ABI, hence it will never match. This test was fixed by instead using the object crate to actually decode the ELF headers looking for the specific e_flags based on reading the RISCV ELF psABI docs.

This PR is best reviewed commit-by-commit, two commits setup the support library for functionality and two commits are for each of the failing run-make tests.

I had to bump the x86_64-gnu-debug job to be ran with a runner with larger disk space.

Part of #132034.

try-job: x86_64-gnu-debug

[WIP] CI: try to drop `--test-args=clang` in `x86_64-gnu-debug` Ok what if I open a separate PR... r? `@ghost` try-job: x86_64-gnu-debug

Full stage 2 build + run-make with full debug info seems to overwhelm the standard 4c runner's storage capacity.

jieyouxu · 2024-10-22T12:00:53Z

What if we downloaded more ~~ram~~ disk space
@bors try

bors · 2024-10-22T12:02:03Z

⌛ Trying commit 03c7f99 with merge 7b157b7...

Run the full stage 2 `run-make` test suite in `x86_64-gnu-debug` Run the full `run-make` test suite in the `x86_64-gnu-debug` CI job. This is currently the *only* CI job where `//@ needs-force-clang-based-test` will be satisfied, so some `run-make` tests will literally never be run otherwise. Before this PR, the CI job only ran `run-make` tests which contains the substring `clang` in its test name, which is both (1) a footgun because it's very easy to forget and (2) it masks tests that would otherwise fail (even failing to compile) because the test is skipped if doesn't have a `clang` in its test name. With the environment of `x86_64-gnu-debug`, two `run-make` tests failed before this PR: 1. `tests/run-make/issue-84395-lto-embed-bitcode/rmake.rs`: this was broken for a long time because `objcopy` in llvm bin tools was renamed to `llvm-objcopy`. This test was converted into a rmake.rs test, rather straight forward. 2. `tests/run-make/cross-lang-lto-riscv-abi/rmake.rs`: this was broken for a long time and never worked. The old version inspected human-readable output of `llvm-readobj --file-header` looking for substring `EF_RISCV_FLOAT_ABI_DOUBLE`, but the human-readable output will only contain something like `Flags: 0x5, RVC, double-float ABI`, hence it will never match. This test was fixed by instead using the `object` crate to actually decode the ELF headers looking for the specific `e_flags` based on reading the RISCV ELF psABI docs. This PR is best reviewed commit-by-commit, two commits setup the support library for functionality and two commits are for each of the failing `run-make` tests. try-job: x86_64-gnu-debug

bors · 2024-10-22T14:05:51Z

☀️ Try build successful - checks-actions
Build commit: 7b157b7 (7b157b7bca4b26d4f40a49c7147d9e3ba8e472fe)

jieyouxu · 2024-10-22T14:11:22Z

try-job passed, so: @rustbot ready

Rolling two reviewers:

r? compiler (for tests and basic support lib changes themselves)
r? infra (for approval to modify runner type)

Kobzol · 2024-10-22T14:13:25Z

+1 on the runner change. If more disk is needed, we don't really have a lot of other options than to just switch to a larger runner..

jieyouxu · 2024-10-23T23:59:00Z

@Kobzol this doesn't specifically need a compiler reviewer, if you think the tests look reasonable then I think it's fine

Kobzol · 2024-10-25T06:58:33Z

Ok! Nice to see one more makefile test removed.

@bors r+

bors · 2024-10-25T06:58:36Z

📌 Commit 03c7f99 has been approved by Kobzol

It is now in the queue for this repository.

bors · 2024-10-25T18:33:21Z

⌛ Testing commit 03c7f99 with merge b188577...

bors · 2024-10-25T20:59:54Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing b188577 to master...

rust-timer · 2024-10-25T22:29:58Z

Finished benchmarking commit (b188577): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary -0.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.1%	[1.1%, 1.1%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.1%	[-1.3%, -0.9%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.4%	[-1.3%, 1.1%]	3

Cycles

Results (secondary -2.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-2.1%, -2.0%]	4
All ❌✅ (primary)	-	-	0